As the world becomes more and more computer dependent and more and more interconnected through computer networks, the need to keep such computer networks up and running increases concomitantly. Congestion in network components can become quite problematic in a variety of scenarios, e.g., where such networks are relied upon for enabling communications, transferring files, coordinating the activities of organizations, etc. There are a number of scenarios such as, for example, virus scanning, checksum calculation, etc., where a server may need to keep incoming data in its runtime memory for pre-processing. Although these activities are typically legitimate functions and oftentimes are required (e.g., to meet service level agreements, ensure the safety of transmissions, etc.), they unfortunately can lead to the server running out of memory, thereby disrupting other applications in the server. These issues are likely to occur when there are large numbers of incoming requests, when requests involve large data files, and/or the like.
There are several solutions available for addressing these concerns. One solution involves denial of service based on a threshold of available memory being surpassed. In this solution, any new incoming request is denied service once the threshold is reached, thereby protecting the total consumed memory from exceeding a predefined limit. Although this solution is effective in some circumstances, it actually can lead to problems in others. For example, multiple requests may be given access at the same or substantially the same time and when the server initially seemingly has sufficient memory to handle them, but these otherwise-allowed requests may gradually consume the server's memory, causing it to run out of memory.
FIG. 1 demonstrates how this denial of service approach to handling congestion can become problematic. The system involved in FIG. 1 is assumed to have a total server working memory of 1,024 MB and a denial of service limit set at 250 MB. Thus, when memory utilization reaches one-quarter of the total working memory, additional requests are denied. Referring to FIG. 1, initially, none of the working memory is used. Three requests are received by the server at the same of substantially the same time, and all are approved for processing, as memory utilization is low. As these requests are being processed, they gradually surpass the memory threshold. At this point, new incoming requests are denied. Yet the requests already being processed continue along and, gradually, they consume the available working memory. At some point, they cause a system crash because the usable system memory is completely consumed.
Another solution for addressing system resource utilization problems involves clustering of servers and/or memory.
In this solution, the server can be scaled across different nodes (e.g., in a clustering-type approach), thereby distributing the load across systems. Here, as soon as the load on a server (calculated on the basis of number of requests/memory, for example), is identified to be high, no more new requests are directed to the server. This process helps balance the load, but a given loaded server instance nonetheless may become overloaded (e.g., as a result of an error in load balancing, as a result of a problem such as the one described above, etc.), thereby resulting in the failure of the processing requests. This approach also sometimes is limited in the sense that the state of the file transfer cannot be resumed in the event of a failure and instead needs to be restarted. This approach also requires separate nodes and/installations to scale, which may not be feasible given monetary constraints and which may be inconvenient in terms of setup and maintenance, for example.
Another solution involves denial of service based on the size of file involved in the transfer (e.g., uploaded to the server). For example, any incoming request is denied if it is associated with a file to be uploaded that has a size larger than a predetermined maximum. One apparent drawback is that there is a limitation regarding size. In one implementation of this solution, whenever there is a connection request, the server assigns a slot of memory (a post box area) based on the memory required by the request from the total available memory to the request. The server gives a denial of service for all other subsequent requests if there are no slots available in the server's main memory. One disadvantage of this implementation, however, includes being unable to address dynamic file uploads as the file sizes of files are not always known at the time of initiation of file transfer. Another disadvantage with this implementation relates to its susceptibility to security threats. For instance, a malicious program could reserve the entire server memory by being deceptive about its size, thereby holding onto the session and obstructing other requests.
Still another solution involves adaptive transfer, where the transfer from the client is adapted based on the network and/or memory load, e.g., using protocols known to the client. One disadvantage to this approach however, is that it typically uses control-level packets to control the transfer from the client, which an external client may not be aware of, leading to a sort of coordination problem and potentially making the approach unworkable.
It will be appreciated that it would be desirable to overcome the above-identified and/or other problems. For example, it will be appreciated that it would be desirable to avert congestion issues at servers (e.g., file transfer servers) such that requests for services (e.g., file transfer services such as, for example, upload and download operations) are able to take place and do not cause a system crash.
One aspect of certain example embodiments relates to overcoming the above-described and/or other issues. For example, one aspect of certain example embodiments relates to techniques for identifying and predicting congestion-related scenarios before they take place, and taking action so that the crashes can be averted, without having to deny service altogether.
Another aspect of certain example embodiments relates to the use of programmed artificial intelligence to understand when congestion-related scenarios are likely to take place. For instance, certain example embodiments may learn strategies based on client identification and understanding client's behavioral patterns, in order to predict congestion in the system.
Another aspect of certain example embodiments relates to learning transfer statistics and creating environment-specific data models that store snapshots of client and/or system behavior(s), using these data models to predict a machine's status, and intelligently handling the clients in a systematic way so as to avert crashes, rather than merely addressing congestion by controlling bandwidth or throttling.
In certain example embodiments, a computer network is provided. The network includes a load balancer and one or more monitors. A cluster comprises a plurality of servers, with each said server including at least one processor, a memory, and a transceiver. The servers are configured to receive, from at least one client computing device remote therefrom and via the load balancer, a plurality of requests to be processed. A plurality of speed limiter modules is included, with each said speed limiter module being associated with a respective one of the servers. The speed limiter modules are configured to at least: receive, from the one or more monitors, current runtime status information pertaining to the computer network and the server associated therewith; predict how a given request will impact the server associated therewith and/or the computer network; determine, based on the received current runtime status information and the prediction, whether an incoming request is to be accepted by the server associated therewith; responsive to a determination that an incoming request is to be accepted, initiate processing of the incoming request via the server associated therewith; determine whether the total resource usage of the server associated therewith exceeds or is likely to exceed an initial resource threshold; and responsive to a determination that the total resource usage of the server associated therewith exceeds or is likely to exceed the initial resource threshold cease acceptance of further incoming requests, and control the speed at which requests currently being processed by the server associated therewith are being handled, such that (a) speeds of all but at least one of the requests currently being processed are throttled, (b) requests that are being throttled are neither terminated nor allowed to timeout nor handed over to another server in the cluster, (c) non-throttled requests are permitted to proceed to completion, and (d) as non-throttled requests are completed, requests that are throttled have their speeds increased in a predetermined manner.
According to certain example embodiments, the requests may be file transfer requests.
According to certain example embodiments, the initial resource threshold may pertain to memory utilization.
According to certain example embodiments, predictions may be made in accordance with a plurality of machine-learned models, e.g., with the the machine-learned models potentially including a first model related to peak operation times, and a second model related to aspects of incoming requests.
According to certain example embodiments, the amount of throttling may be based on current network status and output from the machine-learned models.
In certain example embodiments, a computer network is provided. The computer network comprises one or more monitors. A file transfer server includes at least one processor, a memory, and a transceiver. The file transfer server is configured to receive, via the transceiver and from at least one client computing device remote therefrom, a plurality of file transfer related requests to be processed by the file transfer server. A speed limiter module is configured to at least: receive, from the one or more monitors, current runtime status information pertaining to the computer network and the file transfer server; predict how a given request will impact the file transfer server and/or the computer network; determine, based on the received current runtime status information and the prediction, whether an incoming request is to be accepted by the file transfer server; responsive to a determination that an incoming request is to be accepted, initiate processing of the incoming request via the file transfer server; determine whether the total resource usage of the file transfer server exceeds or is likely to exceed an initial resource threshold; and responsive to a determination that the total resource usage of the file transfer server exceeds or is likely to exceed the initial resource threshold cease acceptance of further incoming requests, and control the speed at which requests currently being processed by the file transfer server are being handled, such that (a) speeds of all but at least one of the requests currently being processed are throttled, (b) requests that are being throttled are neither terminated nor allowed to timeout, (c) non-throttled requests are permitted to proceed to completion, and (d) as non-throttled requests are completed, requests that are throttled have their speeds increased in a predetermined manner.
In certain example embodiments a file transfer server in a computer network is provided. The file transfer server includes at least one processor; a memory; and a transceiver. The transceiver is configured to receive, from at least one client computing device, a plurality of file transfer related requests to be processed by the file transfer server. A speed limiter module is configured to at least: receive, from one or more monitors, current runtime status information pertaining to the computer network and the file transfer server; predict how a given request will impact the file transfer server and/or the computer network; determine, based on the received current runtime status information and the prediction, whether an incoming request is to be accepted by the file transfer server; responsive to a determination that an incoming request is to be accepted, initiate processing of the incoming request via the file transfer server; determine whether the total resource usage of the file transfer server exceeds or is likely to exceed an initial resource threshold; and responsive to a determination that the total resource usage of the file transfer server exceeds or is likely to exceed the initial resource threshold cease acceptance of further incoming requests, and control the speed at which requests currently being processed by the file transfer server are being handled, such that (a) speeds of all but at least one of the requests currently being processed are throttled, (b) requests that are being throttled are neither terminated nor allowed to timeout, (c) non-throttled requests are permitted to proceed to completion, and (d) as non-throttled requests are completed, requests that are throttled have their speeds increased in a predetermined manner.
In addition to the features of the previous paragraphs, counterpart methods, non-transitory computer readable storage media tangibly storing instructions for performing such methods, executable computer programs, and the like, are contemplated herein, as well.
These features, aspects, advantages, and example embodiments may be used separately and/or applied in various combinations to achieve yet further embodiments of this invention.